1 Motivation and goals

In this lab, we will explore some of the data analytical methods needed for the analysis of RNA-Seq data. These cover a wide range of statistical concepts, including

  • hypothesis testing and multiple testing
  • visualization of large matrices using heatmaps
  • clustering and distance metrics
  • ordination methods such as PCA
  • (gene set) enrichment analysis

2 Setup

2.1 Load Packages

First let’s make sure we have all the needed packages installed.

pkgs_needed = c("dplyr", "ggplot2", "DESeq2", "pasilla", "genefilter",
                "pheatmap", "readr", "tibble", 
                "org.Dm.eg.db", "AnnotationDbi", "gsean", "WGCNA")
if (!requireNamespace("BiocManager", quietly = TRUE))
    install.packages("BiocManager")
BiocManager::install(setdiff(pkgs_needed, installed.packages()))

Then let’s load the packages.

library("tidyverse")
library("ggplot2")
library("DESeq2")
library("pasilla")
library("genefilter")
library("pheatmap")

2.2 Example dataset: pasilla

The pasilla data are from an experiment on Drosophila melanogaster cell cultures that investigated the effect of RNAi knock-down of the splicing factor on the cells’ transcriptome. In our case, the data are stored as a rectangular table in a tab-delimited file that comes within the R package pasilla. We use the function read.table to read this file and put the data into the R variable counts.

fn = system.file("extdata", "pasilla_gene_counts.tsv",
                  package = "pasilla", mustWork = TRUE)
countsdf = read.csv(fn, sep = "\t", row.names = "gene_id")
counts = as.matrix(countsdf)

Activity: Use a spreadsheet programme (such as MS Excel) to look at the file.

Question 1: what are the data types of the variables countsdf and counts? What is the difference?

NB: If/when you want to work with other data, you’ll have to prepare (or download) a similar count table and read it into R with similar steps as above. Section 2 of the Bioconductor RNA-Seq workflow gives some useful hints on this; there are also many other online tutorials.

Question 2: what are the dimensions of the counts matrix? Print three random rows from it.

When loading data from a file, a good plausibility check is to print out some of the data, and maybe not only at the very beginning, but also at some random point in the middle, as we have done above.

Question 3: what is the interpretation of counts[45, 2]?

There were two experimental conditions, termed untreated and treated in the header of the count table that we loaded. They correspond to negative control and to siRNA against the gene pasilla, a nuclear RNA binding protein implicated in splicing. The experimental metadata of the 7 samples in this dataset are provided in a spreadsheet-like table. Next, we again use the function system.file to locate a file with this information, which is shipped together with the pasilla package. When you work with your own data, simply prepare and load the corresponding file, or use some other way to generate a dataframe like pasillaSampleAnno.

annotationFile = system.file("extdata", "pasilla_sample_annotation.csv",
                             package = "pasilla", mustWork = TRUE)
pasillaSampleAnno = readr::read_csv(annotationFile)
pasillaSampleAnno
## # A tibble: 7 x 6
##   file  condition type  `number of lane… `total number o…
##   <chr> <chr>     <chr>            <dbl> <chr>           
## 1 trea… treated   sing…                5 35158667        
## 2 trea… treated   pair…                2 12242535 (x2)   
## 3 trea… treated   pair…                2 12443664 (x2)   
## 4 untr… untreated sing…                2 17812866        
## 5 untr… untreated sing…                6 34284521        
## 6 untr… untreated pair…                2 10542625 (x2)   
## 7 untr… untreated pair…                2 12214974 (x2)   
## # … with 1 more variable: `exon counts` <dbl>

As we see here, the overall dataset was produced in two batches, the first one consisting of three sequencing libraries that were subjected to single-read sequencing, the second batch consisting of four libraries for which paired-end sequencing was used. Let’s convert the relevant columns of pasillaSampleAnno into factors, overriding the default level ordering (which is alphabetical) by one that makes more sense to us.

pasillaSampleAnno = mutate(
  pasillaSampleAnno,
  condition = factor(condition, levels = c("untreated", "treated")),
  type      = factor(type, levels = c("single-read", "paired-end")))

Question 4: The design is approximately balanced between the factor of interest, condition, and the nuisance factor type. How can you check that? Use the table function.

We use the constructor function DESeqDataSetFromMatrix to create a DESeqDataSet from the matrix counts and the sample annotation dataframe pasillaSampleAnno.

Note how in the code below, we have to put in extra work to match the column names of the counts object with the file column of the pasillaSampleAnno dataframe, in particular, we need to remove the fb that happens to be used in the file column for some reason. Such data wrangling is very common in bioinformatics and data science. One of the reasons for storing the data in a DESeqDataSet object is that we then no longer have to worry about such things.

mt = match(colnames(counts), sub("fb$", "", pasillaSampleAnno$file))
pasilla = DESeqDataSetFromMatrix(
  countData = counts,
  colData   = pasillaSampleAnno[mt, ],
  design    = ~ condition)
class(pasilla)
## [1] "DESeqDataSet"
## attr(,"package")
## [1] "DESeq2"
is(pasilla, "SummarizedExperiment")
## [1] TRUE

The SummarizedExperiment class –and therefore DESeqDataSet– also contains facilities for storing annotation of the rows of the count matrix. For now, we are content with the gene identifiers from the row names of the counts table.

Question 5: When we constructed our SummarizedExperiment object, we also saved some column metadata which we had initially stored in pasillaSampleAnno. With which function can we extract this information again? (Hint:?SummarizedExperiment)

3 The DESeq2 method

After these preparations, we are now ready to jump straight into differential expression analysis. A choice of standard analysis steps are wrapped into a single function, DESeq.

pasilla = DESeq(pasilla)
## estimating size factors
## estimating dispersions
## gene-wise dispersion estimates
## mean-dispersion relationship
## final dispersion estimates
## fitting model and testing

The DESeq function is simply a wrapper that calls, in order, the functions estimateSizeFactors, estimateDispersions (dispersion estimation) and nbinomWaldTest (hypothesis tests for differential abundance). You can always call these functions individually if you want to modify their behavior or interject custom steps. Let us look at the results (we use the arrange function to order the results by p-value, starting with the lowest).

res = results(pasilla)
arrange(as_tibble(res, rownames = "geneid"), pvalue)
## # A tibble: 14,599 x 7
##    geneid     baseMean log2FoldChange  lfcSE  stat    pvalue      padj
##    <chr>         <dbl>          <dbl>  <dbl> <dbl>     <dbl>     <dbl>
##  1 FBgn00391…     731.          -4.62 0.169  -27.4 4.89e-165 4.07e-161
##  2 FBgn00251…    1501.           2.90 0.127   22.8 1.53e-115 6.38e-112
##  3 FBgn00291…    3706.          -2.20 0.0970 -22.7 1.33e-113 3.69e-110
##  4 FBgn00033…    4343.          -3.18 0.144  -22.2 9.56e-109 1.99e-105
##  5 FBgn00350…     638.          -2.56 0.137  -18.6 1.29e- 77 2.14e- 74
##  6 FBgn00398…     262.          -4.16 0.233  -17.9 1.26e- 71 1.74e- 68
##  7 FBgn00347…     226.          -3.51 0.215  -16.4 3.86e- 60 4.59e- 57
##  8 FBgn00298…     490.          -2.45 0.152  -16.1 2.92e- 58 3.03e- 55
##  9 FBgn00000…     342.           2.68 0.183   14.7 9.51e- 49 8.79e- 46
## 10 FBgn00510…     153.           2.33 0.177   13.2 1.31e- 39 1.09e- 36
## # … with 14,589 more rows

4 Explore the result

Question 6: Plot the data for the top 2 genes (those with the smallest p-values), as well as for 6 random genes

5 The histogram of p-values and multiple testing

Question 7: Plot the histogram of p-values.

The distribution displays two main components: a uniform background with values between 0 and 1, and a peak of small p-values at the left. The uniform background corresponds to the non-differentially expressed genes. Usually this is the majority of genes. The left hand peak corresponds to differentially expressed genes.

The ratio of the level of the background to the height of the peak gives us a rough indication of the false discovery rate (FDR) that would be associated with calling the genes in the leftmost bin differentially expressed.

Question 8:

How many p-values are \(\le 0.01\)? Compute the median height of all the bins in the histogram, and divide this by the height of the first (leftmost) bin. What is an interpretation of this quantity? Compare it to the false discovery rate as computed by the Benjamini-Hochberg method.

6 MA plot

Read the Wikipedia description for MA plots. The plots shows the observed fold change versus the mean of the (size-factor normalized) counts. Logarithmic scaling is used for both axes. Points which fall out of the y-axis range are plotted as triangles. To produce an MA plot for our data, we can use the function plotMA in the DESeq2 package.

plotMA(pasilla, ylim = c( -2, 2))

7 PCA plot

Question 9: Use the DESeq2 function plotPCA to produce a two-dimensional ordination of the 7 samples in the dataset. Before doing that, first transform the data with the variance stabilizing transformation provided by DESeq2.

This type of plot is useful for visualizing the overall effect of experimental covariates and/or to detect batch effects. Here, the first principal axis, PC1, is mostly aligned with the experimental covariate of interest (untreated / treated), while the second axis is roughly aligned with the sequencing protocol (single-read / paired-end). Instead of PCA, other ordination methods, for instance multi-dimensional scaling, can also be useful.

8 Heatmaps

Draw a heatmap of the transformed data pas_trsf. Since it’s impractical to show all 14599 rows, only plot the subset of the 30 most variable genes.

Question 10:

If you want, you can try a different heatmap package (for example ComplexHeatmap) and explore a more enriched heatmap plot.

9 Two-factor analysis

Besides the treatment with siRNA, the pasilla data have another covariate, type, which indicates the type of sequencing that was performed. We saw in the PCA plot that this type had a considerable systematic effect on the data. Our basic analysis did not take this account, but we will do so now. This should help us get a more correct picture of which differences in the data are attributable to the treatment, and which are confounded—or masked—by the sequencing type.

pasillaTwoFactor = pasilla
design(pasillaTwoFactor) = formula(~ type + condition)
pasillaTwoFactor = DESeq(pasillaTwoFactor)

Of the two variables type and condition, the one of primary interest is the latter, and in DESeq2, the convention is to put it at the end of the formula. This convention has no effect on the model fitting, but it helps simplify some of the subsequent results reporting. Again, we access the results using the results function.

res2 = results(pasillaTwoFactor)
arrange(as_tibble(res2, rownames = "geneid"), pvalue)
## # A tibble: 14,599 x 7
##    geneid     baseMean log2FoldChange  lfcSE  stat    pvalue      padj
##    <chr>         <dbl>          <dbl>  <dbl> <dbl>     <dbl>     <dbl>
##  1 FBgn00033…    4343.          -3.13 0.109  -28.7 1.78e-181 1.57e-177
##  2 FBgn00265…   43909.          -2.48 0.0875 -28.3 2.04e-176 8.98e-173
##  3 FBgn00391…     731.          -4.62 0.167  -27.7 2.69e-169 7.88e-166
##  4 FBgn00251…    1501.           2.85 0.104   27.4 5.38e-165 1.18e-161
##  5 FBgn00291…    3706.          -2.20 0.0961 -22.9 1.27e-115 2.23e-112
##  6 FBgn00350…     638.          -2.58 0.129  -20.0 7.55e- 89 1.11e- 85
##  7 FBgn00398…     262.          -4.18 0.217  -19.3 5.54e- 83 6.97e- 80
##  8 FBgn00347…     226.          -3.54 0.206  -17.2 2.75e- 66 3.03e- 63
##  9 FBgn00298…     490.          -2.44 0.148  -16.5 2.13e- 61 2.08e- 58
## 10 FBgn00000…     342.           2.62 0.159   16.5 2.74e- 61 2.41e- 58
## # … with 14,589 more rows

It is also possible to retrieve the \(\log_2\) fold changes, p-values and adjusted p-values associated with the type variable. The function results takes an argument contrast that lets users specify the name of the variable, the level that corresponds to the numerator of the fold change and the level that corresponds to the denominator of the fold change.

resType = results(pasillaTwoFactor, 
                  contrast = c("type", "single-read", "paired-end"))

arrange(as_tibble(resType, rownames = "geneid"), pvalue)
## # A tibble: 14,599 x 7
##    geneid    baseMean log2FoldChange  lfcSE   stat    pvalue      padj
##    <chr>        <dbl>          <dbl>  <dbl>  <dbl>     <dbl>     <dbl>
##  1 FBgn0003…   6604.           -2.55 0.0989 -25.8  3.89e-147 3.33e-143
##  2 FBgn0053…    295.           -2.60 0.178  -14.6  2.23e- 48 9.53e- 45
##  3 FBgn0083…     62.7          -4.65 0.426  -10.9  1.11e- 27 2.41e- 24
##  4 FBgn0029…   1611.           -3.58 0.329  -10.9  1.13e- 27 2.41e- 24
##  5 FBgn0086…   2753.           -1.06 0.0999 -10.6  3.10e- 26 5.30e- 23
##  6 FBgn0028…   1731.            1.19 0.115   10.4  3.97e- 25 5.67e- 22
##  7 FBgn0030…    128.           -2.20 0.224   -9.83 8.54e- 23 1.04e- 19
##  8 FBgn0039…    593.            1.10 0.119    9.26 2.00e- 20 2.07e- 17
##  9 FBgn0037…    491.            1.67 0.181    9.25 2.17e- 20 2.07e- 17
## 10 FBgn0053…    139.          -11.0  1.20    -9.12 7.52e- 20 6.43e- 17
## # … with 14,589 more rows

So what did we gain from this analysis that took into account type as a nuisance factor (sometimes also called, more politely, a blocking factor), compared to the simple comparison between two groups?

Question 11: Count and compare the number of genes that pass a certain significance threshold in each of the two analyses.

Question 12: Make a scatterplot of -log10 of the p-values from both analyses against each other. What do you notice?

10 A gene set enrichment analysis

We here conduct a basic workflow the purpose of which is to give us some feeling or intuition about the results. This is not hardcore statistics. There are numerous options, subtleties and various software implementations with a wide range of quality.

First, we need to embark in one of the favourite bioinformatics pasttimes, converting gene identifies from one system to another. The function that we use here, gsean from the equinymous package, emits some warnings, which we could dig into, or ignore for now.

library("gsean")
library("org.Dm.eg.db")

statistic = setNames(res2$stat, rownames(res2))
geneid = AnnotationDbi::select(org.Dm.eg.db, names(statistic), 
                               "ENTREZID", "FLYBASE")
exprs_pasilla = counts(pasilla, normalized = TRUE)
stopifnot(identical(geneid$FLYBASE, names(statistic)), 
          identical(geneid$FLYBASE, rownames(exprs_pasilla)))

names(statistic) = rownames(exprs_pasilla) = geneid$ENTREZID

load(system.file("data", "GO_dme.rda", package = "gsean"))
gsea = gsean(GO_dme, statistic, exprs_pasilla)
p = GSEA.barplot(gsea, category = 'pathway', score = 'NES', 
             top = 15, pvalue = 'padj', sort = 'padj', 
             numChar = 110) + 
  theme(plot.margin = margin(10, 10, 10, 50))
plotly::ggplotly(p)
LS0tCnRpdGxlOiAiRWxlbWVudHMgb2YgUk5BLVNlcSBkYXRhIGFuYWx5c2lzIgpzdWJ0aXRsZTogIkV4ZXJjaXNlIGZvciB0aGUgY291cnNlIERhdGEgQW5hbHlzaXMgZm9yIHRoZSBMaWZlIFNjaWVuY2VzIGF0IHRoZSBVbml2ZXJzaXR5IG9mIEhlaWRlbGJlcmciCmRhdGU6ICIyMDIwLTA2LTA1IChEYXkgMTMpIgphdXRob3I6ICJCcml0dGEgVmVsdGVuIGFuZCBXb2xmZ2FuZyBIdWJlciIKb3V0cHV0OiAKICBCaW9jU3R5bGU6Omh0bWxfZG9jdW1lbnQ6CiAgICB0b2M6IHRydWUKICAgIHRvY19mbG9hdDogdHJ1ZQogICAgY29kZV9kb3dubG9hZDogdHJ1ZQotLS0KCjwhLS0KVGhpcyBsYWIgaXMgbWVhbnQgdG8gcmVuZGVyIGludG8gdHdvIEhUTUwgZmlsZXMuIENsaWNraW5nICpLbml0KiBvbiBpdCBpbiBSU3R1ZGlvIHdpbGwgcHJvZHVjZSBUZXN0aW5nLWFuZC1STkFzZXEuaHRtbCAod2hpY2ggY29udGFpbnMgdGhlIFF1ZXN0aW9ucyBhbmQgQW5zd2VycykgYXMgd2VsbCBhcyBUZXN0aW5nLWFuZC1STkFzZXEtbm9hbnMuUm1kLiBPbiB0aGUgbGF0dGVyLCBwbGVhc2UgcnVuICpLbml0KiBvciBybWFya2Rvd246OnJlbmRlciBhZ2FpbiB0byBvYnRhaW4gVGVzdGluZy1hbmQtUk5BLW5vYW5zLmh0bWwgKHdoaWNoIGNvbnRhaW5zIG9ubHkgdGhlIFF1ZXN0aW9ucyBhbmQgbm8gQW5zd2VycykuCi0tPgoKYGBge3Igc2V0dXAsIGluY2x1ZGU9RkFMU0V9CmtuaXRyOjpvcHRzX2NodW5rJHNldChlY2hvID0gVFJVRSwgZmlnLmRpbSA9IGMoKDErc3FydCg1KSkvMiwgMSkgKiA1LCBjYWNoZSA9IFRSVUUsIGF1dG9kZXAgPSBUUlVFKSAKb3B0aW9ucyh3aWR0aCA9IDcwKQpgYGAKCiMgTW90aXZhdGlvbiBhbmQgZ29hbHMKCkluIHRoaXMgbGFiLCB3ZSB3aWxsIGV4cGxvcmUgc29tZSBvZiB0aGUgZGF0YSBhbmFseXRpY2FsIG1ldGhvZHMgbmVlZGVkIGZvciB0aGUgYW5hbHlzaXMgb2YgUk5BLVNlcSBkYXRhLiBUaGVzZSBjb3ZlciBhIHdpZGUgcmFuZ2Ugb2Ygc3RhdGlzdGljYWwgY29uY2VwdHMsIGluY2x1ZGluZwoKLSBoeXBvdGhlc2lzIHRlc3RpbmcgYW5kIG11bHRpcGxlIHRlc3RpbmcKLSB2aXN1YWxpemF0aW9uIG9mIGxhcmdlIG1hdHJpY2VzIHVzaW5nIGhlYXRtYXBzCi0gY2x1c3RlcmluZyBhbmQgZGlzdGFuY2UgbWV0cmljcwotIG9yZGluYXRpb24gbWV0aG9kcyBzdWNoIGFzIFBDQQotIChnZW5lIHNldCkgZW5yaWNobWVudCBhbmFseXNpcwoKCiMgU2V0dXAKCiMjIExvYWQgUGFja2FnZXMKCkZpcnN0IGxldCdzIG1ha2Ugc3VyZSB3ZSBoYXZlIGFsbCB0aGUgbmVlZGVkIHBhY2thZ2VzIGluc3RhbGxlZC4KCmBgYHtyIGluc3RhbGwsIGV2YWwgPSBGQUxTRX0KcGtnc19uZWVkZWQgPSBjKCJkcGx5ciIsICJnZ3Bsb3QyIiwgIkRFU2VxMiIsICJwYXNpbGxhIiwgImdlbmVmaWx0ZXIiLAogICAgICAgICAgICAgICAgInBoZWF0bWFwIiwgInJlYWRyIiwgInRpYmJsZSIsIAogICAgICAgICAgICAgICAgIm9yZy5EbS5lZy5kYiIsICJBbm5vdGF0aW9uRGJpIiwgImdzZWFuIiwgIldHQ05BIikKaWYgKCFyZXF1aXJlTmFtZXNwYWNlKCJCaW9jTWFuYWdlciIsIHF1aWV0bHkgPSBUUlVFKSkKICAgIGluc3RhbGwucGFja2FnZXMoIkJpb2NNYW5hZ2VyIikKQmlvY01hbmFnZXI6Omluc3RhbGwoc2V0ZGlmZihwa2dzX25lZWRlZCwgaW5zdGFsbGVkLnBhY2thZ2VzKCkpKQpgYGAKClRoZW4gbGV0J3MgbG9hZCB0aGUgcGFja2FnZXMuCgpgYGB7ciBsb2FkLCB3YXJuaW5nID0gRkFMU0UsIG1lc3NhZ2UgPSBGQUxTRX0KbGlicmFyeSgidGlkeXZlcnNlIikKbGlicmFyeSgiZ2dwbG90MiIpCmxpYnJhcnkoIkRFU2VxMiIpCmxpYnJhcnkoInBhc2lsbGEiKQpsaWJyYXJ5KCJnZW5lZmlsdGVyIikKbGlicmFyeSgicGhlYXRtYXAiKQpgYGAKCiMjIEV4YW1wbGUgZGF0YXNldDogcGFzaWxsYQoKVGhlIGBwYXNpbGxhYCBkYXRhIGFyZSBmcm9tIGFuIGV4cGVyaW1lbnQgb24gRHJvc29waGlsYSBtZWxhbm9nYXN0ZXIgY2VsbApjdWx0dXJlcyB0aGF0IGludmVzdGlnYXRlZCB0aGUgZWZmZWN0IG9mIFJOQWkga25vY2stZG93biBvZiB0aGUgc3BsaWNpbmcgZmFjdG9yIApvbiB0aGUgY2VsbHMnIHRyYW5zY3JpcHRvbWUuIEluIG91ciBjYXNlLCB0aGUgZGF0YSBhcmUgc3RvcmVkIGFzIGEgcmVjdGFuZ3VsYXIgdGFibGUgCmluIGEgdGFiLWRlbGltaXRlZCBmaWxlIHRoYXQgY29tZXMgd2l0aGluIHRoZSBSIHBhY2thZ2UgYHBhc2lsbGFgLiBXZSB1c2UgdGhlIGZ1bmN0aW9uIApgcmVhZC50YWJsZWAgdG8gcmVhZCB0aGlzIGZpbGUgYW5kIHB1dCB0aGUgZGF0YSBpbnRvIHRoZSBSIHZhcmlhYmxlIGBjb3VudHNgLgoKYGBge3IgbG9hZHBhcywgcmVzdWx0cyA9ICJoaWRlIiwgZXJyb3IgPSBGQUxTRX0KZm4gPSBzeXN0ZW0uZmlsZSgiZXh0ZGF0YSIsICJwYXNpbGxhX2dlbmVfY291bnRzLnRzdiIsCiAgICAgICAgICAgICAgICAgIHBhY2thZ2UgPSAicGFzaWxsYSIsIG11c3RXb3JrID0gVFJVRSkKY291bnRzZGYgPSByZWFkLmNzdihmbiwgc2VwID0gIlx0Iiwgcm93Lm5hbWVzID0gImdlbmVfaWQiKQpjb3VudHMgPSBhcy5tYXRyaXgoY291bnRzZGYpCmBgYAoKKipBY3Rpdml0eSoqOiBVc2UgYSBzcHJlYWRzaGVldCBwcm9ncmFtbWUgKHN1Y2ggYXMgTVMgRXhjZWwpIHRvIGxvb2sgYXQgdGhlIGZpbGUuCgpgYGB7ciBxdWVzbnVtLCBlY2hvID0gRkFMU0V9CmlxdWVzID0gMApgYGAKCioqUXVlc3Rpb24gYHIgKGlxdWVzID0gaXF1ZXMrMSlgKio6IHdoYXQgYXJlIHRoZSBkYXRhIHR5cGVzIG9mIHRoZSB2YXJpYWJsZXMgYGNvdW50c2RmYCBhbmQgYGNvdW50c2A/IFdoYXQgaXMgdGhlIGRpZmZlcmVuY2U/CgoKTkI6IElmL3doZW4geW91IHdhbnQgdG8gd29yayB3aXRoIG90aGVyIGRhdGEsIHlvdSdsbCBoYXZlIHRvIHByZXBhcmUgKG9yIGRvd25sb2FkKSBhIHNpbWlsYXIgY291bnQgdGFibGUgYW5kIHJlYWQgaXQgaW50byBSIHdpdGggc2ltaWxhciBzdGVwcyBhcyBhYm92ZS4gU2VjdGlvbiAyIG9mIHRoZSBbQmlvY29uZHVjdG9yIFJOQS1TZXEgd29ya2Zsb3ddKGh0dHBzOi8vYmlvY29uZHVjdG9yLm9yZy9wYWNrYWdlcy9yZWxlYXNlL3dvcmtmbG93cy92aWduZXR0ZXMvcm5hc2VxR2VuZS9pbnN0L2RvYy9ybmFzZXFHZW5lLmh0bWwpIGdpdmVzIHNvbWUgdXNlZnVsIGhpbnRzIG9uIHRoaXM7IHRoZXJlIGFyZSBhbHNvIG1hbnkgb3RoZXIgb25saW5lIHR1dG9yaWFscy4KCioqUXVlc3Rpb24gYHIgKGlxdWVzID0gaXF1ZXMrMSlgKio6IHdoYXQgYXJlIHRoZSBkaW1lbnNpb25zIG9mIHRoZSBgY291bnRzYCBtYXRyaXg/IFByaW50IHRocmVlIHJhbmRvbSByb3dzIGZyb20gaXQuCgoKV2hlbiBsb2FkaW5nIGRhdGEgZnJvbSBhIGZpbGUsIGEgZ29vZCBwbGF1c2liaWxpdHkgY2hlY2sgaXMgdG8gcHJpbnQgb3V0IHNvbWUgb2YgCnRoZSBkYXRhLCBhbmQgbWF5YmUgbm90IG9ubHkgYXQgdGhlIHZlcnkgYmVnaW5uaW5nLCBidXQgYWxzbyBhdCBzb21lIHJhbmRvbSAKcG9pbnQgaW4gdGhlIG1pZGRsZSwgYXMgd2UgaGF2ZSBkb25lIGFib3ZlLiAKCioqUXVlc3Rpb24gYHIgKGlxdWVzID0gaXF1ZXMrMSlgKio6IHdoYXQgaXMgdGhlIGludGVycHJldGF0aW9uIG9mIGBjb3VudHNbNDUsIDJdYD8KCgpUaGVyZSB3ZXJlIHR3byBleHBlcmltZW50YWwgY29uZGl0aW9ucywgdGVybWVkICoqdW50cmVhdGVkKiogYW5kICoqdHJlYXRlZCoqIGluIAp0aGUgaGVhZGVyIG9mIHRoZSBjb3VudCB0YWJsZSB0aGF0IHdlIGxvYWRlZC4gVGhleSBjb3JyZXNwb25kIHRvIG5lZ2F0aXZlCmNvbnRyb2wgYW5kIHRvIHNpUk5BIGFnYWluc3QgdGhlIGdlbmUgcGFzaWxsYSwgYSBudWNsZWFyIFJOQSBiaW5kaW5nIHByb3RlaW4gaW1wbGljYXRlZCBpbiBzcGxpY2luZy4gIFRoZSBleHBlcmltZW50YWwgbWV0YWRhdGEgb2YgdGhlIApgciBuY29sKGNvdW50cylgIHNhbXBsZXMgaW4gdGhpcyBkYXRhc2V0IGFyZSBwcm92aWRlZCBpbiBhIHNwcmVhZHNoZWV0LWxpa2UgCnRhYmxlLiBOZXh0LCB3ZSBhZ2FpbiB1c2UgdGhlIGZ1bmN0aW9uIGBzeXN0ZW0uZmlsZWAgdG8gbG9jYXRlIGEgZmlsZSB3aXRoIHRoaXMgaW5mb3JtYXRpb24sIHdoaWNoIGlzIApzaGlwcGVkIHRvZ2V0aGVyIHdpdGggdGhlIGBwYXNpbGxhYCBwYWNrYWdlLiBXaGVuIHlvdSB3b3JrIHdpdGggeW91ciBvd24gZGF0YSwgCnNpbXBseSBwcmVwYXJlIGFuZCBsb2FkIHRoZSBjb3JyZXNwb25kaW5nIGZpbGUsIG9yIHVzZSBzb21lIG90aGVyIHdheSB0byAKZ2VuZXJhdGUgYSBkYXRhZnJhbWUgbGlrZSBgcGFzaWxsYVNhbXBsZUFubm9gLgoKYGBge3IgYW5ub3RhdGlvbkZpbGUsIG1lc3NhZ2UgPSBGQUxTRX0KYW5ub3RhdGlvbkZpbGUgPSBzeXN0ZW0uZmlsZSgiZXh0ZGF0YSIsICJwYXNpbGxhX3NhbXBsZV9hbm5vdGF0aW9uLmNzdiIsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgcGFja2FnZSA9ICJwYXNpbGxhIiwgbXVzdFdvcmsgPSBUUlVFKQpwYXNpbGxhU2FtcGxlQW5ubyA9IHJlYWRyOjpyZWFkX2Nzdihhbm5vdGF0aW9uRmlsZSkKcGFzaWxsYVNhbXBsZUFubm8KYGBgCgpBcyB3ZSBzZWUgaGVyZSwgdGhlIG92ZXJhbGwgZGF0YXNldCB3YXMgcHJvZHVjZWQgaW4gdHdvIGJhdGNoZXMsIHRoZSBmaXJzdCBvbmUgCmNvbnNpc3Rpbmcgb2YgdGhyZWUgc2VxdWVuY2luZyBsaWJyYXJpZXMgdGhhdCB3ZXJlIHN1YmplY3RlZCB0byBzaW5nbGUtcmVhZCAKc2VxdWVuY2luZywgdGhlIHNlY29uZCBiYXRjaCBjb25zaXN0aW5nIG9mIGZvdXIgbGlicmFyaWVzIGZvciB3aGljaCBwYWlyZWQtZW5kIApzZXF1ZW5jaW5nIHdhcyB1c2VkLiAgTGV0J3MgY29udmVydCB0aGUgcmVsZXZhbnQgY29sdW1ucyBvZiBgcGFzaWxsYVNhbXBsZUFubm9gIAppbnRvIGZhY3RvcnMsIG92ZXJyaWRpbmcgdGhlIGRlZmF1bHQgbGV2ZWwgb3JkZXJpbmcgKHdoaWNoIGlzIGFscGhhYmV0aWNhbCkgYnkgCm9uZSB0aGF0IG1ha2VzIG1vcmUgc2Vuc2UgdG8gdXMuCgpgYGB7ciBmYWN0b3JzfQpwYXNpbGxhU2FtcGxlQW5ubyA9IG11dGF0ZSgKICBwYXNpbGxhU2FtcGxlQW5ubywKICBjb25kaXRpb24gPSBmYWN0b3IoY29uZGl0aW9uLCBsZXZlbHMgPSBjKCJ1bnRyZWF0ZWQiLCAidHJlYXRlZCIpKSwKICB0eXBlICAgICAgPSBmYWN0b3IodHlwZSwgbGV2ZWxzID0gYygic2luZ2xlLXJlYWQiLCAicGFpcmVkLWVuZCIpKSkKYGBgCgoqKlF1ZXN0aW9uIGByIChpcXVlcyA9IGlxdWVzKzEpYCoqOiBUaGUgZGVzaWduIGlzIGFwcHJveGltYXRlbHkgYmFsYW5jZWQgYmV0d2VlbiB0aGUgZmFjdG9yIG9mIGludGVyZXN0LCBgY29uZGl0aW9uYCwgYW5kIHRoZSBudWlzYW5jZSBmYWN0b3IgYHR5cGVgLiBIb3cgY2FuIHlvdSBjaGVjayB0aGF0PyBVc2UgdGhlIGB0YWJsZWAgZnVuY3Rpb24uCgoKV2UgdXNlIHRoZSBjb25zdHJ1Y3RvciBmdW5jdGlvbiBgREVTZXFEYXRhU2V0RnJvbU1hdHJpeGAgdG8gY3JlYXRlIGEgYERFU2VxRGF0YVNldGAgZnJvbSB0aGUgbWF0cml4IGBjb3VudHNgIGFuZCB0aGUgc2FtcGxlIGFubm90YXRpb24gZGF0YWZyYW1lIGBwYXNpbGxhU2FtcGxlQW5ub2AuCgpOb3RlIGhvdyBpbiB0aGUgY29kZSBiZWxvdywgd2UgaGF2ZSB0byBwdXQgaW4gZXh0cmEgd29yayB0byBtYXRjaCB0aGUgY29sdW1uIG5hbWVzIG9mIHRoZSBgY291bnRzYCBvYmplY3Qgd2l0aCB0aGUgYGZpbGVgIGNvbHVtbiBvZiB0aGUgYHBhc2lsbGFTYW1wbGVBbm5vYCBkYXRhZnJhbWUsIGluIHBhcnRpY3VsYXIsIHdlIG5lZWQgdG8gcmVtb3ZlIHRoZSBgZmJgIHRoYXQgaGFwcGVucyB0byBiZSB1c2VkIGluIHRoZSBgZmlsZWAgY29sdW1uIGZvciBzb21lIHJlYXNvbi4gU3VjaCBkYXRhIHdyYW5nbGluZyBpcyB2ZXJ5IGNvbW1vbiBpbiBiaW9pbmZvcm1hdGljcyBhbmQgZGF0YSBzY2llbmNlLiBPbmUgb2YgdGhlIHJlYXNvbnMgZm9yIHN0b3JpbmcgdGhlIGRhdGEgaW4gYSBgREVTZXFEYXRhU2V0YCBvYmplY3QgaXMgdGhhdCB3ZSB0aGVuIG5vIGxvbmdlciBoYXZlIHRvIHdvcnJ5IGFib3V0IHN1Y2ggdGhpbmdzLgoKYGBge3IgREVTZXEyLCBtZXNzYWdlID0gRkFMU0UsIHdhcm5pbmcgPSBGQUxTRX0KbXQgPSBtYXRjaChjb2xuYW1lcyhjb3VudHMpLCBzdWIoImZiJCIsICIiLCBwYXNpbGxhU2FtcGxlQW5ubyRmaWxlKSkKcGFzaWxsYSA9IERFU2VxRGF0YVNldEZyb21NYXRyaXgoCiAgY291bnREYXRhID0gY291bnRzLAogIGNvbERhdGEgICA9IHBhc2lsbGFTYW1wbGVBbm5vW210LCBdLAogIGRlc2lnbiAgICA9IH4gY29uZGl0aW9uKQpjbGFzcyhwYXNpbGxhKQppcyhwYXNpbGxhLCAiU3VtbWFyaXplZEV4cGVyaW1lbnQiKQpgYGAKClRoZSBgU3VtbWFyaXplZEV4cGVyaW1lbnRgIGNsYXNzIC0tYW5kIHRoZXJlZm9yZSBgREVTZXFEYXRhU2V0YC0tIGFsc28KY29udGFpbnMgZmFjaWxpdGllcyBmb3Igc3RvcmluZyBhbm5vdGF0aW9uIG9mIHRoZSByb3dzIG9mIHRoZSBjb3VudCBtYXRyaXguIApGb3Igbm93LCB3ZSBhcmUgY29udGVudCB3aXRoIHRoZSBnZW5lIGlkZW50aWZpZXJzIGZyb20gdGhlIHJvdyBuYW1lcyBvZiAKdGhlIGBjb3VudHNgIHRhYmxlLgoKKipRdWVzdGlvbiBgciAoaXF1ZXMgPSBpcXVlcysxKWAqKjogV2hlbiB3ZSBjb25zdHJ1Y3RlZCBvdXIgYFN1bW1hcml6ZWRFeHBlcmltZW50YCBvYmplY3QsIHdlIAphbHNvIHNhdmVkIHNvbWUgY29sdW1uIG1ldGFkYXRhIHdoaWNoIHdlIGhhZCBpbml0aWFsbHkgc3RvcmVkIGluIApgcGFzaWxsYVNhbXBsZUFubm9gLiBXaXRoIHdoaWNoIGZ1bmN0aW9uIGNhbiB3ZSBleHRyYWN0IHRoaXMgaW5mb3JtYXRpb24gYWdhaW4/CihIaW50OmA/U3VtbWFyaXplZEV4cGVyaW1lbnRgKQoKCiMgVGhlIERFU2VxMiBtZXRob2QKCkFmdGVyIHRoZXNlIHByZXBhcmF0aW9ucywgd2UgYXJlIG5vdyByZWFkeSB0byBqdW1wIHN0cmFpZ2h0IGludG8gZGlmZmVyZW50aWFsIApleHByZXNzaW9uIGFuYWx5c2lzLiBBIGNob2ljZSBvZiBzdGFuZGFyZCBhbmFseXNpcyBzdGVwcyBhcmUgd3JhcHBlZCBpbnRvIGEKc2luZ2xlIGZ1bmN0aW9uLCBgREVTZXFgLgoKYGBge3IgZGVzZXEsIG1lc3NhZ2UgPSBUUlVFfQpwYXNpbGxhID0gREVTZXEocGFzaWxsYSkKYGBgCgpUaGUgYERFU2VxYCBmdW5jdGlvbiBpcyBzaW1wbHkgYSB3cmFwcGVyIHRoYXQgY2FsbHMsIGluIG9yZGVyLCB0aGUgZnVuY3Rpb25zIApgZXN0aW1hdGVTaXplRmFjdG9yc2AsIGBlc3RpbWF0ZURpc3BlcnNpb25zYCAoZGlzcGVyc2lvbiBlc3RpbWF0aW9uKSBhbmQgCmBuYmlub21XYWxkVGVzdGAgKGh5cG90aGVzaXMgdGVzdHMgZm9yIGRpZmZlcmVudGlhbCBhYnVuZGFuY2UpLiBZb3UgY2FuCmFsd2F5cyBjYWxsIHRoZXNlIGZ1bmN0aW9ucyBpbmRpdmlkdWFsbHkgaWYgeW91IHdhbnQgdG8gbW9kaWZ5IHRoZWlyIGJlaGF2aW9yCm9yIGludGVyamVjdCBjdXN0b20gc3RlcHMuIExldCB1cyBsb29rIGF0IHRoZSByZXN1bHRzICh3ZSB1c2UgdGhlIGBhcnJhbmdlYCAKZnVuY3Rpb24gdG8gb3JkZXIgdGhlIHJlc3VsdHMgYnkgcC12YWx1ZSwgc3RhcnRpbmcgd2l0aCB0aGUgbG93ZXN0KS4KCmBgYHtyIHRoZXJlc3VsdHN9CnJlcyA9IHJlc3VsdHMocGFzaWxsYSkKYXJyYW5nZShhc190aWJibGUocmVzLCByb3duYW1lcyA9ICJnZW5laWQiKSwgcHZhbHVlKQpgYGAKCiMgRXhwbG9yZSB0aGUgcmVzdWx0CgoqKlF1ZXN0aW9uIGByIChpcXVlcyA9IGlxdWVzKzEpYCoqOiAKUGxvdCB0aGUgZGF0YSBmb3IgdGhlIHRvcCAyIGdlbmVzICh0aG9zZSB3aXRoIHRoZSBzbWFsbGVzdCBwLXZhbHVlcyksIGFzIHdlbGwgYXMgZm9yIDYgcmFuZG9tIGdlbmVzCgoKIyBUaGUgaGlzdG9ncmFtIG9mIHAtdmFsdWVzIGFuZCBtdWx0aXBsZSB0ZXN0aW5nCgoqKlF1ZXN0aW9uIGByIChpcXVlcyA9IGlxdWVzKzEpYCoqOiAKUGxvdCB0aGUgaGlzdG9ncmFtIG9mIHAtdmFsdWVzLgoKICAKVGhlIGRpc3RyaWJ1dGlvbiBkaXNwbGF5cyB0d28gbWFpbiBjb21wb25lbnRzOiBhIHVuaWZvcm0gYmFja2dyb3VuZCB3aXRoIHZhbHVlcyAKYmV0d2VlbiAwIGFuZCAxLCBhbmQgYSBwZWFrIG9mIHNtYWxsIHAtdmFsdWVzIGF0IHRoZSBsZWZ0LiAgVGhlIHVuaWZvcm0gCmJhY2tncm91bmQgY29ycmVzcG9uZHMgdG8gdGhlIG5vbi1kaWZmZXJlbnRpYWxseSBleHByZXNzZWQgZ2VuZXMuIFVzdWFsbHkgdGhpcyAKaXMgdGhlIG1ham9yaXR5IG9mIGdlbmVzLiBUaGUgbGVmdCBoYW5kIHBlYWsgY29ycmVzcG9uZHMgdG8gZGlmZmVyZW50aWFsbHkgZXhwcmVzc2VkIGdlbmVzLgoKVGhlIHJhdGlvIG9mIHRoZSBsZXZlbCBvZiB0aGUgYmFja2dyb3VuZCB0byB0aGUgaGVpZ2h0IG9mIHRoZSBwZWFrIGdpdmVzIHVzIAphIHJvdWdoIGluZGljYXRpb24gb2YgdGhlIGZhbHNlIGRpc2NvdmVyeSByYXRlIChGRFIpIHRoYXQgd291bGQgYmUgYXNzb2NpYXRlZCAKd2l0aCBjYWxsaW5nIHRoZSBnZW5lcyBpbiB0aGUgbGVmdG1vc3QgYmluIGRpZmZlcmVudGlhbGx5IGV4cHJlc3NlZC4KCioqUXVlc3Rpb24gYHIgKGlxdWVzID0gaXF1ZXMrMSlgKio6IAoKSG93IG1hbnkgcC12YWx1ZXMgYXJlICRcbGUgMC4wMSQ/IENvbXB1dGUgdGhlIG1lZGlhbiBoZWlnaHQgb2YgYWxsIHRoZSBiaW5zIGluIHRoZSBoaXN0b2dyYW0sIGFuZCBkaXZpZGUgdGhpcyBieSB0aGUgaGVpZ2h0IG9mIHRoZSBmaXJzdCAobGVmdG1vc3QpIGJpbi4gV2hhdCBpcyBhbiBpbnRlcnByZXRhdGlvbiBvZiB0aGlzIHF1YW50aXR5PyBDb21wYXJlIGl0IHRvIHRoZSBmYWxzZSBkaXNjb3ZlcnkgcmF0ZSBhcyBjb21wdXRlZCBieSB0aGUgQmVuamFtaW5pLUhvY2hiZXJnIG1ldGhvZC4KCiAgCgojIE1BIHBsb3QKClJlYWQgdGhlIFdpa2lwZWRpYSBkZXNjcmlwdGlvbiBmb3IgW01BIHBsb3RzXShodHRwczovL2VuLndpa2lwZWRpYS5vcmcvd2lraS9NQV9wbG90KS4gVGhlIHBsb3RzIHNob3dzIHRoZSBvYnNlcnZlZCBmb2xkIGNoYW5nZSB2ZXJzdXMgdGhlIG1lYW4gb2YgdGhlIChzaXplLWZhY3RvciBub3JtYWxpemVkKSBjb3VudHMuIExvZ2FyaXRobWljIHNjYWxpbmcgaXMgdXNlZCBmb3IgYm90aCBheGVzLiBQb2ludHMgd2hpY2ggZmFsbCBvdXQgb2YgdGhlIHktYXhpcyByYW5nZSBhcmUgcGxvdHRlZCBhcyB0cmlhbmdsZXMuIFRvIHByb2R1Y2UgYW4gTUEgcGxvdCBmb3Igb3VyIGRhdGEsIHdlIGNhbiB1c2UgdGhlIGZ1bmN0aW9uIGBwbG90TUFgIGluIHRoZSBgREVTZXEyYCBwYWNrYWdlLgoKYGBge3IgTUF9CnBsb3RNQShwYXNpbGxhLCB5bGltID0gYyggLTIsIDIpKQpgYGAKCiMgUENBIHBsb3QKCioqUXVlc3Rpb24gYHIgKGlxdWVzID0gaXF1ZXMrMSlgKio6IApVc2UgdGhlIGBERVNlcTJgIGZ1bmN0aW9uIGBwbG90UENBYCB0byBwcm9kdWNlIGEgdHdvLWRpbWVuc2lvbmFsIG9yZGluYXRpb24gb2YgdGhlIGByIG5jb2wocGFzaWxsYSlgIHNhbXBsZXMgaW4gdGhlIGRhdGFzZXQuIEJlZm9yZSBkb2luZyB0aGF0LCBmaXJzdCB0cmFuc2Zvcm0gdGhlIGRhdGEgd2l0aCB0aGUgdmFyaWFuY2Ugc3RhYmlsaXppbmcgdHJhbnNmb3JtYXRpb24gcHJvdmlkZWQgYnkgYERFU2VxMmAuIAoKClRoaXMgdHlwZSBvZiBwbG90IGlzIHVzZWZ1bCBmb3IgdmlzdWFsaXppbmcgdGhlIG92ZXJhbGwgZWZmZWN0IG9mIGV4cGVyaW1lbnRhbCAKY292YXJpYXRlcyBhbmQvb3IgdG8gZGV0ZWN0IGJhdGNoIGVmZmVjdHMuIEhlcmUsIHRoZSBmaXJzdCBwcmluY2lwYWwgYXhpcywKUEMxLCBpcyBtb3N0bHkgYWxpZ25lZCB3aXRoIHRoZSBleHBlcmltZW50YWwgY292YXJpYXRlIG9mIGludGVyZXN0IAoodW50cmVhdGVkIC8gdHJlYXRlZCksIHdoaWxlIHRoZSBzZWNvbmQgYXhpcyBpcyByb3VnaGx5IGFsaWduZWQgd2l0aCAKdGhlIHNlcXVlbmNpbmcgcHJvdG9jb2wgKHNpbmdsZS1yZWFkIC8gcGFpcmVkLWVuZCkuIEluc3RlYWQgb2YgUENBLCBvdGhlciAKb3JkaW5hdGlvbiBtZXRob2RzLCBmb3IgaW5zdGFuY2UgbXVsdGktZGltZW5zaW9uYWwgc2NhbGluZywgY2FuIGFsc28gYmUgdXNlZnVsLgoKIyBIZWF0bWFwcwoKRHJhdyBhIGhlYXRtYXAgb2YgdGhlIHRyYW5zZm9ybWVkIGRhdGEgYHBhc190cnNmYC4gU2luY2UgaXQncyBpbXByYWN0aWNhbCB0byBzaG93IGFsbCBgciBucm93KHBhc2lsbGEpYCByb3dzLCBvbmx5IHBsb3QgdGhlIHN1YnNldCBvZiB0aGUgMzAgbW9zdCB2YXJpYWJsZSBnZW5lcy4KCioqUXVlc3Rpb24gYHIgKGlxdWVzID0gaXF1ZXMrMSlgKio6IAogIAogIApJZiB5b3Ugd2FudCwgeW91IGNhbiB0cnkgYSBkaWZmZXJlbnQgaGVhdG1hcCBwYWNrYWdlIChmb3IgZXhhbXBsZSBgQ29tcGxleEhlYXRtYXBgKSBhbmQgZXhwbG9yZSBhIG1vcmUgZW5yaWNoZWQgaGVhdG1hcCBwbG90LgoKIyBUd28tZmFjdG9yIGFuYWx5c2lzCgpCZXNpZGVzIHRoZSB0cmVhdG1lbnQgd2l0aCBzaVJOQSwgdGhlIGBwYXNpbGxhYCBkYXRhIGhhdmUgYW5vdGhlciBjb3ZhcmlhdGUsCmB0eXBlYCwgd2hpY2ggaW5kaWNhdGVzIHRoZSB0eXBlIG9mIHNlcXVlbmNpbmcgdGhhdCB3YXMgcGVyZm9ybWVkLgpXZSBzYXcgaW4gdGhlIFBDQSBwbG90IHRoYXQgdGhpcyBgdHlwZWAgaGFkIGEgY29uc2lkZXJhYmxlIApzeXN0ZW1hdGljIGVmZmVjdCBvbiB0aGUgZGF0YS4gT3VyIGJhc2ljIGFuYWx5c2lzIGRpZCBub3QgdGFrZSB0aGlzIGFjY291bnQsIApidXQgd2Ugd2lsbCBkbyBzbyBub3cuIFRoaXMgc2hvdWxkIGhlbHAgdXMgZ2V0IGEgbW9yZSBjb3JyZWN0IHBpY3R1cmUgb2Ygd2hpY2gKZGlmZmVyZW5jZXMgaW4gdGhlIGRhdGEgYXJlIGF0dHJpYnV0YWJsZSB0byB0aGUgdHJlYXRtZW50LCBhbmQgd2hpY2ggYXJlCmNvbmZvdW5kZWQtLS1vciBtYXNrZWQtLS1ieSB0aGUgc2VxdWVuY2luZyB0eXBlLgoKYGBge3IgcmVwbGFjZURlc2lnbiwgbWVzc2FnZSA9IEZBTFNFLCByZXN1bHRzID0gImhpZGUifQpwYXNpbGxhVHdvRmFjdG9yID0gcGFzaWxsYQpkZXNpZ24ocGFzaWxsYVR3b0ZhY3RvcikgPSBmb3JtdWxhKH4gdHlwZSArIGNvbmRpdGlvbikKcGFzaWxsYVR3b0ZhY3RvciA9IERFU2VxKHBhc2lsbGFUd29GYWN0b3IpCmBgYAoKT2YgdGhlIHR3byB2YXJpYWJsZXMgYHR5cGVgIGFuZCBgY29uZGl0aW9uYCwgdGhlIG9uZSBvZiBwcmltYXJ5IGludGVyZXN0CmlzIHRoZSBsYXR0ZXIsIGFuZCBpbiBgREVTZXEyYCwgdGhlIGNvbnZlbnRpb24gaXMgdG8gcHV0IGl0IGF0IHRoZSBlbmQgb2YgdGhlCmZvcm11bGEuIFRoaXMgY29udmVudGlvbiBoYXMgbm8gZWZmZWN0IG9uIHRoZSBtb2RlbCBmaXR0aW5nLCBidXQgaXQgaGVscHMgCnNpbXBsaWZ5IHNvbWUgb2YgdGhlIHN1YnNlcXVlbnQgcmVzdWx0cyByZXBvcnRpbmcuIEFnYWluLCB3ZSBhY2Nlc3MgdGhlIHJlc3VsdHMgCnVzaW5nIHRoZSBgcmVzdWx0c2AgZnVuY3Rpb24uCgpgYGB7ciBtdWx0aVJlc3VsdHN9CnJlczIgPSByZXN1bHRzKHBhc2lsbGFUd29GYWN0b3IpCmFycmFuZ2UoYXNfdGliYmxlKHJlczIsIHJvd25hbWVzID0gImdlbmVpZCIpLCBwdmFsdWUpCmBgYAoKSXQgaXMgYWxzbyBwb3NzaWJsZSB0byByZXRyaWV2ZSB0aGUgJFxsb2dfMiQgZm9sZCBjaGFuZ2VzLCBwLXZhbHVlcyBhbmQgYWRqdXN0ZWQKcC12YWx1ZXMgYXNzb2NpYXRlZCB3aXRoIHRoZSBgdHlwZWAgdmFyaWFibGUuICBUaGUgZnVuY3Rpb24gYHJlc3VsdHNgIHRha2VzIGFuCmFyZ3VtZW50IGBjb250cmFzdGAgdGhhdCBsZXRzIHVzZXJzIHNwZWNpZnkgdGhlIG5hbWUgb2YgdGhlIHZhcmlhYmxlLCB0aGUgbGV2ZWwKdGhhdCBjb3JyZXNwb25kcyB0byB0aGUgbnVtZXJhdG9yIG9mIHRoZSBmb2xkIGNoYW5nZSBhbmQgdGhlIGxldmVsIHRoYXQgY29ycmVzcG9uZHMKdG8gdGhlIGRlbm9taW5hdG9yIG9mIHRoZSBmb2xkIGNoYW5nZS4KCmBgYHtyIG11bHRpVHlwZVJlc3VsdHN9CnJlc1R5cGUgPSByZXN1bHRzKHBhc2lsbGFUd29GYWN0b3IsIAogICAgICAgICAgICAgICAgICBjb250cmFzdCA9IGMoInR5cGUiLCAic2luZ2xlLXJlYWQiLCAicGFpcmVkLWVuZCIpKQoKYXJyYW5nZShhc190aWJibGUocmVzVHlwZSwgcm93bmFtZXMgPSAiZ2VuZWlkIiksIHB2YWx1ZSkKYGBgCgpTbyB3aGF0IGRpZCB3ZSBnYWluIGZyb20gdGhpcyBhbmFseXNpcyB0aGF0IHRvb2sgaW50byBhY2NvdW50IGB0eXBlYCBhcyBhIApudWlzYW5jZSBmYWN0b3IgKHNvbWV0aW1lcyBhbHNvIGNhbGxlZCwgbW9yZSBwb2xpdGVseSwgYSBibG9ja2luZyBmYWN0b3IpLCAKY29tcGFyZWQgdG8gdGhlIHNpbXBsZSBjb21wYXJpc29uIGJldHdlZW4gdHdvIGdyb3Vwcz8gCgoqKlF1ZXN0aW9uIGByIChpcXVlcyA9IGlxdWVzKzEpYCoqOiAKQ291bnQgYW5kIGNvbXBhcmUgdGhlIG51bWJlciBvZiBnZW5lcyB0aGF0IHBhc3MgYSBjZXJ0YWluIHNpZ25pZmljYW5jZSB0aHJlc2hvbGQgaW4gCmVhY2ggb2YgdGhlIHR3byBhbmFseXNlcy4KCgoqKlF1ZXN0aW9uIGByIChpcXVlcyA9IGlxdWVzKzEpYCoqOiAKTWFrZSBhIHNjYXR0ZXJwbG90IG9mIC1sb2cxMCBvZiB0aGUgcC12YWx1ZXMgZnJvbSBib3RoIGFuYWx5c2VzIGFnYWluc3QgZWFjaCBvdGhlci4gV2hhdCBkbyB5b3Ugbm90aWNlPwogIAoKIyBBIGdlbmUgc2V0IGVucmljaG1lbnQgYW5hbHlzaXMKCldlIGhlcmUgY29uZHVjdCBhIGJhc2ljIHdvcmtmbG93IHRoZSBwdXJwb3NlIG9mIHdoaWNoIGlzIHRvIGdpdmUgdXMgc29tZSBmZWVsaW5nIG9yIGludHVpdGlvbiBhYm91dCB0aGUgcmVzdWx0cy4gVGhpcyBpcyBub3QgaGFyZGNvcmUgc3RhdGlzdGljcy4gVGhlcmUgYXJlIG51bWVyb3VzIG9wdGlvbnMsIHN1YnRsZXRpZXMgYW5kIHZhcmlvdXMgc29mdHdhcmUgaW1wbGVtZW50YXRpb25zIHdpdGggYSB3aWRlIHJhbmdlIG9mIHF1YWxpdHkuIAoKRmlyc3QsIHdlIG5lZWQgdG8gZW1iYXJrIGluIG9uZSBvZiB0aGUgZmF2b3VyaXRlIGJpb2luZm9ybWF0aWNzIHBhc3R0aW1lcywgY29udmVydGluZyBnZW5lIGlkZW50aWZpZXMgZnJvbSBvbmUgc3lzdGVtIHRvIGFub3RoZXIuIFRoZSBmdW5jdGlvbiB0aGF0IHdlIHVzZSBoZXJlLCBgZ3NlYW5gIGZyb20gdGhlIGVxdWlueW1vdXMgcGFja2FnZSwgZW1pdHMgc29tZSB3YXJuaW5ncywgd2hpY2ggd2UgY291bGQgZGlnIGludG8sIG9yIGlnbm9yZSBmb3Igbm93LgoKYGBge3Igb3JnZG0sIG1lc3NhZ2UgPSBGQUxTRSwgd2FybmluZyA9IEZBTFNFLCByZXN1bHRzID0gImhpZGUifQpsaWJyYXJ5KCJnc2VhbiIpCmxpYnJhcnkoIm9yZy5EbS5lZy5kYiIpCgpzdGF0aXN0aWMgPSBzZXROYW1lcyhyZXMyJHN0YXQsIHJvd25hbWVzKHJlczIpKQpnZW5laWQgPSBBbm5vdGF0aW9uRGJpOjpzZWxlY3Qob3JnLkRtLmVnLmRiLCBuYW1lcyhzdGF0aXN0aWMpLCAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICJFTlRSRVpJRCIsICJGTFlCQVNFIikKZXhwcnNfcGFzaWxsYSA9IGNvdW50cyhwYXNpbGxhLCBub3JtYWxpemVkID0gVFJVRSkKc3RvcGlmbm90KGlkZW50aWNhbChnZW5laWQkRkxZQkFTRSwgbmFtZXMoc3RhdGlzdGljKSksIAogICAgICAgICAgaWRlbnRpY2FsKGdlbmVpZCRGTFlCQVNFLCByb3duYW1lcyhleHByc19wYXNpbGxhKSkpCgpuYW1lcyhzdGF0aXN0aWMpID0gcm93bmFtZXMoZXhwcnNfcGFzaWxsYSkgPSBnZW5laWQkRU5UUkVaSUQKCmxvYWQoc3lzdGVtLmZpbGUoImRhdGEiLCAiR09fZG1lLnJkYSIsIHBhY2thZ2UgPSAiZ3NlYW4iKSkKZ3NlYSA9IGdzZWFuKEdPX2RtZSwgc3RhdGlzdGljLCBleHByc19wYXNpbGxhKQpwID0gR1NFQS5iYXJwbG90KGdzZWEsIGNhdGVnb3J5ID0gJ3BhdGh3YXknLCBzY29yZSA9ICdORVMnLCAKICAgICAgICAgICAgIHRvcCA9IDE1LCBwdmFsdWUgPSAncGFkaicsIHNvcnQgPSAncGFkaicsIAogICAgICAgICAgICAgbnVtQ2hhciA9IDExMCkgKyAKICB0aGVtZShwbG90Lm1hcmdpbiA9IG1hcmdpbigxMCwgMTAsIDEwLCA1MCkpCmBgYApgYGB7ciBnc2VhcGxvdCwgZmlnLmRpbSA9IGMoMTAsIDMuNSl9CnBsb3RseTo6Z2dwbG90bHkocCkKYGBgCgoKYGBge3Igd2l0aG91dHNvbHV0aW9ucywgZWNobyA9IEZBTFNFfQpyZW1vdmVBbnN3ZXJzID0gZnVuY3Rpb24oeCwgYmVnaW4gPSAiXjxkaXYgY2xhc3M9XCJhbnN3ZXJcIj4kIiwgZW5kID0gIl48L2Rpdj4kIikgewogIGkxID0gZ3JlcChiZWdpbiwgeCkKICBpMiA9IGdyZXAoZW5kLCB4KQogIHN0b3BpZm5vdChsZW5ndGgoaTEpPT1sZW5ndGgoaTIpLCBhbGwoaTE8aTIpKQogIHIgPSB1bmxpc3QobWFwcGx5KGA6YCwgaTEsIGkyLCBTSU1QTElGWSAgPSBGQUxTRSkpIAogIGlmIChsZW5ndGgocikgPiAwKSB4Wy1yXSBlbHNlIHgKfQpmaWxlcyA9IGMoIlRlc3RpbmctYW5kLVJOQXNlcS5SbWQiLCAiVGVzdGluZy1hbmQtUk5BLW5vYW5zLlJtZCIpCnJlYWRMaW5lcyhmaWxlc1sxXSkgJT4lIAogIHJlbW92ZUFuc3dlcnMgJT4lCiAgd3JpdGVMaW5lcyhmaWxlc1syXSkKIyBybWFya2Rvd246OnJlbmRlcihmaWxlc1syXSkKYGBgCgojIyBMaXRlcmF0dXJlCgpbTW9kZXJuIFN0YXRpc3RpY3MgZm9yIE1vZGVybiBCaW9sb2d5IGJ5IFN1c2FuIEhvbG1lcyBhbmQgV29sZmdhbmcgSHViZXIuIENoYXB0ZXIgODogSGlnaC1UaHJvdWdocHV0IENvdW50IERhdGFdKGh0dHBzOi8vd3d3Lmh1YmVyLmVtYmwuZGUvbXNtYi9DaGFwLUNvdW50RGF0YS5odG1sKS4K